Method and apparatus for informing user of image recognition error in imaging system

ABSTRACT

Disclosed is a method and an apparatus for informing a user of an image recognition error in an imaging system. The method includes detecting environmental factors causing errors of image recognition when image recognition is requested by the user, calculating analysis indices corresponding to the environmental factors, perceiving whether image recognition is suitably performed by checking whether the analysis indices are included in a normal range of predetermined reference values, and informing the user of the suitability or the unsuitability of image recognition.

PRIORITY

This application claims the benefit under 35 U.S.C. §119(a) of anapplication entitled “Method And Apparatus For Informing User Of ImageRecognition Error In Imaging System” filed in the Korean IndustrialProperty Office on Sep. 22, 2006 and assigned Serial No. 2006-92448, thecontents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to image processing, and moreparticularly to a method and an apparatus for informing a user of causesof errors incurred in the process of image recognition by an imagingsystem.

2. Description of the Related Art

Generally, image recognition establishes with image data aregion-of-interest including feature image data needed for imagerecognition, segments image data corresponding to a setregion-of-interest, compares segmented image data with previously storedimage data for image recognition, and performs a function of imagerecognition.

FIG. 1 the conventional image recognition. In step 101, if imagerecognition is requested by a user, then an imaging system receivesimage data which includes an object of recognition, and proceeds to step103. If image recognition is not requested by the user, the procedurerepeats step 101. In step 103, the imaging system eliminates noise ofthe image data, and adjusts the luminosity of the image data to besuitable for image recognition. In step 105, the imaging system searchesthe image data for the region-of-interest including the object ofrecognition, segments the picture image by an amount of a searchedregion-of-interest, and extracts feature image data needed for imagerecognition from a segmented picture image. In step 107, the imagesystem performs image recognition using the feature image data of inputimage data. In more detail, the image system extracts feature image datafrom the image data which has been previously stored. Then the imagesystem compares the feature image data of input image data with thefeature image data of previously stored image data, and searches for thepreviously stored image data which is most similar to the feature imagedata of input image data. The image system recognizes a user imagecorresponding to the found image data as the image of a user which hasrequested image recognition.

As described above with reference to FIG. 1, whether the result of imagerecognition is either successful or unsuccessful, the conventionalimaging system does not inform the user of the relevant state thereof.

In particular, when the imaging system results in an unsuccessful imagerecognition, it is problematic that the imaging system does not informthe user of a cause of the failure in the image recognition.

SUMMARY OF THE INVENTION

Accordingly, the present invention has been made to solve the aboveproblems occurring in the prior art, and it is an object of the presentinvention to provide a method and an apparatus for perceiving a cause ofan image recognition error and for informing a user of the imagerecognition error when the error is incurred in the process of imagerecognition.

In order to accomplish this object, there is provided a method forinforming a user of an image recognition error in an imaging system,including generating image data by taking a picture of an object ofrecognition if image recognition is requested by a user, calculatinganalysis indices corresponding to environmental factors by detectingfrom the image data the environmental factors that may cause imagerecognition errors, determining whether the analysis indices aresuitable for image recognition by checking whether each of the analysisindices is included in or excluded from a normal range of referencevalues, and informing the user of the cause of the image recognitionerror after perceiving that each of the environmental factorscorresponding to the analysis indices excluded from the normal range ofthe reference values is a cause of the image recognition error if any ofthe analysis indices are excluded from the normal range of the referencevalues.

According to the present invention, there is provided an apparatus forinforming a user of an image recognition error in an imaging system,including an image analyzing unit for calculating analysis indicescorresponding to environmental factors by detecting from input imagedata the environmental factors that may cause the image recognitionerrors, for outputting a cause of the image recognition error afterperceiving that each of the environmental factors corresponding to theanalysis indices excluded from the normal range of reference values isthe cause of the image recognition error if any of the analysis indicesare excluded from the normal range of the predetermined referencevalues, and for outputting the image data to an image processing unit ifall of the analysis indices are included in the normal range of thepredetermined reference values, an image processing unit for searchingpreviously stored image data for image data including an object ofrecognition of the image data, for calculating a recognition reliabilitycorresponding to the reliability of an outcome resulting from comparingthe searched image data with the image data, and for outputting thecause of the recognition error after rechecking the cause thereofthrough the analysis indices if the recognition reliability is less thana reference reliability, and a control unit for controlling the imageanalyzing unit and the image processing unit if image recognition isrequested by the user, and for informing the user of the cause of therecognition error provided by the image analyzing unit or by the imageprocessing unit.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features, and advantages of the presentinvention will be more apparent from the following detailed descriptiontaken in conjunction with the accompanying drawings, in which:

FIG. 1 illustrates an operation of image recognition of a conventionalimaging system;

FIG. 2 illustrates an imaging system according to the present invention;

FIG. 3 illustrates an image processing unit and an image analyzing unitincluded in the imaging system according to the present invention;

FIG. 4 illustrates an operation for performing a function of imagerecognition with image data according to the present invention;

FIG. 5 illustrates an operation for informing a user of a cause of animage recognition error by perceiving the cause of the image recognitionerror with the image data of according to the present invention;

FIG. 6 illustrates the conception of a picture image of a face accordingto the present invention; and

FIG. 7 illustrates a calculation of each reference value needed forchecking whether image recognition can be suitably performed accordingto the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, preferred embodiments of the present invention will bedescribed with reference to the accompanying drawings. In the followingdescription, the same elements will be designated by the same referencenumerals although they are shown in different drawings. Further, in thefollowing description of the present invention, a detailed descriptionof known functions and configurations incorporated herein will beomitted for the sake of clarity and conciseness.

When an image recognition error is incurred in the present invention, animaging system perceives the cause of the image recognition error, andinforms a user of the cause. The imaging system according to the presentinvention also informs the user of the cause of the recognition errorincurred by an environmental factor among the causes of the recognitionerrors.

The environmental factors imply external causes which have influences onimage recognition of image data, and includes for example, anillumination state and a size state of the image data, and an anglestate of an object of recognition. More specifically, when theillumination state of the image data is brighter than a first value ordarker than a second value, the imaging system cannot perform imagerecognition. Also when the size state of the image data is smaller thana prescribed size, the imaging system cannot perform image recognition.Furthermore, the imaging system cannot perform image recognition byenvironmental factors other than the states of illumination, size of theimage data, and the angle state of the object of recognition, either.

To be more recognizable the environmental factors can be limited to thefactors such as the states of illumination, size of the image data, andthe angle state of the object of recognition. It will now be consideredthat the imaging system of the present invention informs the user thatthe environmental factors cause the image recognition errors.

FIG. 2 illustrates an imaging system according to the present invention.Referring to FIG. 2, a description will be given to the operations ofthe units configuring the imaging system of the present invention.

The imaging system includes a control unit 201, a communication unit203, an image processing unit 205, an image analyzing unit 207, a cameraunit 209, a memory unit 211, a display unit 213 and a key input unit(not shown).

The communication unit 203 performs a communication function with anyexternal apparatus. Particularly, the communication unit 203 transmitsthe image data to any external apparatus or receives the image data fromany external apparatus. Under the control of the control unit 201, thecamera unit 209 takes a picture of an inanimate or an animate object togenerate the image data. Under the control of control unit 201, thedisplay unit 213 displays a current state and an operation state of theimaging system and the image data, and generally can be constructed witha Liquid Crystal Display (LCD) or Organic Light Emitting Diodes (OLED).The key input unit (not shown) includes numeric keys, including a * keyand a # key, for image processing, and other various types of well-knownfunction keys, and generates a key signal in response to a key pressedby the user to provide to the control unit 201.

The memory unit 211 preserves data required to control the imagingsystem. In particular, the memory unit 211 stores learning data forcalculating reference values required to check whether image recognitioncan be suitably performed, and recognition data required to perform afunction of image recognition.

The learning data is for calculating the reference values required todetermine whether image recognition can be suitably performed. Forinstance, the learning data can be a set of arbitrary image data, andcan be a set of arbitrary feature image data which is extracted from theimage data. In addition, the learning data can be classified intoanalysis learning data and reliability learning data. Herein, theanalysis learning data can be classified into a set of recognitionsuitable data and a set of recognition unsuitable data. The reliabilitylearning data can be classified into a set of reliability suitable dataand a set of reliability unsuitable data.

A set of recognition suitable data is data that has been alreadyperceived to be suitable for image recognition by an image recognizingunit 307 (Refer to FIG. 3). Furthermore, A set of recognition suitabledata can be arbitrary image data, and can be also feature image dataextracted from the arbitrary image data. A set of recognition unsuitabledata is a set of data that has been previously perceived to beunsuitable for image recognition by an image recognizing unit 307.Furthermore, a set of recognition unsuitable data can be arbitrary imagedata, and can also be feature image data extracted from the arbitraryimage data.

A set of reliability suitable data is data that has been previouslyperceived to be suitable for image recognition by an image recognizingunit 307 and by an image recognition analyzing unit 319 (Refer to FIG.3). Furthermore, set of reliability suitable data can be arbitrary imagedata, and can also be feature image data extracted from the arbitraryimage data. A set of reliability unsuitable data is data that has beenpreviously perceived to be suitable for image recognition by an imagerecognition analyzing unit 319 but has been previously perceived to beunsuitable for image recognition by an image recognizing unit 307.Furthermore, set of reliability unsuitable data can be arbitrary imagedata, and can be also feature image data which is extracted from thearbitrary image data.

When the image recognizing unit 307 performs a function of imagerecognition, the recognition data is compared with the feature imagedata of the image data which is input to the image recognizing unit 307.Thus, the recognition data can be arbitrary image data or feature imagedata of the arbitrary image data. Further, when the recognition of aface is performed during image recognition, the recognition data can beregistered by the user.

The image processing unit 205 preprocesses the image data in order toperform a process of image recognition, searches for aregion-of-interest including the object of recognition, and outputs tothe control unit 201 image data formed by separating only a partcorresponding to a searched region-of-interest from the image data. Whenthe image data is provided by the control unit 201, the image processingunit 205 performs the function of image recognition. Then, the imageanalyzing unit 207 analyzes the image data provided by the control unit201, and perceives whether the image data is suitable for imagerecognition.

FIG. 3 illustrates the image processing unit and the image analyzingunit included in the imaging system according to the present invention.Referring to FIG. 3, components which constitute the image processingunit 205 and the image analyzing unit 207 will be described.

The image processing unit 205 includes a preprocessing unit 301, animage segmenting unit 303, a feature extracting unit 305, an imagerecognizing unit 307 and an image recognition reliability learning unit309. The preprocessing unit 301 processes the image data provided by thecontrol unit 201 to eliminate noise from the image data, or to adjustthe luminosity of the image data to be suitable for the recognition, andoutputs the preprocessed image data to the image segmenting unit 303.The image segmenting unit 303 searches the image data for theregion-of-interest including the object of recognition, and segments theimage data including only the searched region-of-interest. The imagesegmenting unit 303 outputs the segmented image data to the control unit201.

The feature extracting unit 305 receives the segmented image data fromthe control unit 201, extracts the feature image data which ischaracteristic information required to distinguish the object ofrecognition, and outputs an extracted feature image data to the imagerecognizing unit 307. The image recognizing unit 307 receives thefeature image data, and performs a function of image recognition.Specifically, the image recognizing unit 307 searches for therecognition data, and extracts the feature image data of the image datacorresponding to the recognition data. At this time, the imagerecognizing unit 307 compares input feature image data with the featureimage data which has just been extracted, and calculates the recognitionreliability by using an outcome resulting from the comparison. Further,the image recognizing unit 307 compares the recognition reliability witha reference reliability set by the image recognition reliabilitylearning unit 309. When the recognition reliability is equal to orgreater than the reference reliability, the image recognizing unit 307stores a recognition result, and can perform the remaining functionsconnected with image recognition. When the recognition reliability isless than the reference reliability, the image recognizing unit 307perceives the cause of the recognition error, and informs the user ofthe perceived cause.

The reference reliability is a value required to determine whether imagerecognition is successfully performed after the function of the imagerecognition has been performed. If the recognition reliability is equalto or greater than the reference reliability, the image recognizing unit307 perceives that image recognition has been successfully performed. Ifthe recognition reliability is less than the reference reliability, theimage recognizing unit 307 perceives that image recognition has beenunsuccessfully performed.

The image recognition reliability learning unit 309 sets the referencereliability with the reliability learning data among the learning datawhich has been previously stored. For example, the image recognitionreliability learning unit 309 can set up the reference reliability byusing a method of maximum likelihood.

The image analyzing unit 207 includes an illumination analyzing unit311, an angle analyzing unit 313, a size analyzing unit 315, an imageanalysis learning unit 317 and an image recognition analyzing unit 319.

The illumination analyzing unit 311, angle analyzing unit 313 and sizeanalyzing unit 315 calculate analysis indices before the function ofimage recognition is to be performed. Analysis indices representrequisites which must be analyzed in order to perceive whether the imagedata is suitable for image recognition before the function of imagerecognition is to be performed. According to a preferred embodiment ofthe present invention, the number of the analysis indices is a total often as follows: a luminosity mean, a luminosity variance, ablack-and-white pixel percentage, a segmentation luminosity mean, asegmentation luminosity variance, a segmented black-and-white pixelpercentage, the number of segmented pixels, a minutia segment length, aminutia segment ratio and a minutia angle.

The illumination analyzing unit 311 receives the image data or segmentedimage data from the control unit 201, and analyzes the illuminationstate. At this time, the illumination analyzing unit 311 is now able toanalyze the illumination state of the image data by calculatingpercentages of the luminosity mean, the luminosity variance and thenumber of black-and-white pixels of the image data. Upon receiving thesegmented image data, the illumination analyzing unit 311 can analyzethe illumination state of the segmented image data by calculatingpercentages of the segmentation luminosity mean, the segmentationluminosity variance and the number of segmented black-and-white pixelsof the segmented image data. Upon receiving the segmented image data,the size analyzing unit 315 analyzes the size state of the segmentedimage data. Then, the size analyzing unit 315 is now able to analyze thesize state of the segmented image data by calculating the number ofimage pixels of the segmented image data.

Upon receiving the segmented image data, the angle analyzing unit 313analyzes the angle state of the object of recognition. At this time, theangle analyzing unit 313 searches for the minutiae of the segmentedimage data, and can analyze the angle state of the object of recognitionwith coordinates of a searched minutia. Considering when the angle stateis analyzed in a face recognition, which is a type of image recognition,the angle analyzing unit 313 calculates the coordinates of the eyes andmouth on the face, and can calculate the minutia segment length, theminutia segment ratio, and the minutia angle by using the coordinates ofthe eyes and mouth.

The image analysis learning unit 317 sets the reference values requiredto perceive whether image recognition can be suitably performed beforethe function of image recognition is to be performed in the analysislearning data which has been previously stored. Specifically, the imageanalysis learning unit 317 sets a minimum luminosity mean, a maximumluminosity mean, a reference luminosity variance, and a reference pixelpercentage required for comparison between them and the analysis indexof the illumination state of the image data. Further, the Image analysislearning unit 317 sets a minimum segmentation luminosity mean, a maximumsegmentation luminosity mean, a reference segmentation luminosityvariance, and a reference segmented pixel percentage required forcomparison between them and the analysis index of the illumination stateof the segmented image data. Subsequently, the image analysis learningunit 317 sets the number of reference pixels required for comparisonbetween it and the analysis index of the size state of the segmentedimage data. Next, the image analysis learning unit 317 sets up areference minutia segment length, a minimum minutia segment ratio, amaximum minutia segment ratio, and a reference minutia angle, requiredfor comparison between them and the analysis index of the angle state ofthe object of recognition of the segmented image data.

The image recognition analyzing unit 319 searches for the referencevalues calculated by the image analyzing learning unit 317 and for theanalysis indices calculated by the illumination analyzing unit 311, bythe angle analyzing unit 313, and by the size analyzing unit 315. Theimage recognition analyzing unit 319 then compares the analysis indiceswith the reference values according to requisites for image recognition,and perceives whether the analysis indices are suitable for imagerecognition. Herein, if any one of the analysis indices fail to meet therequisites for image recognition, the image recognition analyzing unit319 perceives that the environmental factors, corresponding to theanalysis indices which fail to meet the requisites, are the causes ofthe recognition errors and informs the user of the causes.

FIG. 7 illustrates that reference values are set up with a set ofanalysis learning data according to the present invention. Withreference to FIGS. 1 to 7, a procedure in which the analysis learningunit 317 sets the reference values of the analysis indices will bedescribed as follows.

The learning data is needed to calculate the reference values requiredto determine whether image recognition can be suitably performed. Forexample, the learning data can be a set of arbitrary image data, or aset of arbitrary feature image data extracted from the image data. Inaddition, the learning data can be classified into analysis learningdata and reliability learning data. Herein, the analysis learning datacan be classified into a set of recognition suitable data and a set ofrecognition unsuitable data. The reliability learning data can beclassified into a set of reliability suitable data and a set ofreliability unsuitable data.

A set of the recognition suitable data is data that the imagerecognizing unit 307 has previously suitable data can be arbitrary imagedata, or feature image data which is extracted from the arbitrary imagedata. A set of the recognition unsuitable data is a set of data that theimage recognition unit 307 has previously perceived to be unsuitable forimage recognition. In addition, a set of the recognition unsuitable datacan be arbitrary image data, or feature image data extracted from thearbitrary image data.

A set of the reliability suitable data is data that has been previouslyperceived to be suitable for image recognition by the image recognitionanalyzing unit 319 and the image recognizing unit 307, and alsocorresponds to data having a reliability is greater than a referencereliability. A set of the reliability suitable data can be arbitraryimage data, or feature image data extracted from the arbitrary imagedata. A set of the reliability unsuitable data is data that the imagerecognition analyzing unit 319 has previously perceived to be suitablefor image recognition or a set of data that the image recognizing unit307 has previously perceived to be unsuitable for image recognition, andwhose reliability that is less than a reference reliability. Herein, aset of the reliability unsuitable data can be arbitrary image data, orfeature image data extracted from the arbitrary image data.

Particularly, after performing a process of image recognition, based ona result of image recognition, the analysis learning data stores theimage data in the memory unit 211 to be able to update it. Specifically,after performing the process of image recognition, if the results aresuccessful, relevant image data can be stored in the memory unit 211 asthe recognition suitable data. If the results are unsuccessful, relevantimage data can be stored in the memory unit 211 as the recognitionunsuitable data.

After performing the process of image recognition, based on areliability of the image data, the reliability learning data stores theimage data in the memory unit 211 to update the data.

It will now be considered that a process of setting a reference valuehas been performed before the user requests image recognition, and thatthe learning data corresponds to a set of the image data.

The control unit 201 controls the image analysis learning unit 317 ofthe image analyzing unit 207 to calculate the reference values requiredto perceive whether image recognition can be suitably performed, andsets calculated reference values.

FIG. 7 illustrates a calculation of each reference value needed forchecking whether image recognition can be suitably performed accordingto the present invention. Hereinafter, with reference to FIG. 7, adescription will be given to the process where the image analysislearning unit 317 calculates the reference values.

In step 701, the image analysis learning unit 317 (refer to FIG. 3)searches for a set of the recognition suitable data which has beenpreviously preserved. Furthermore, the image analysis learning unit 317calculates the analysis indices by using all of the image data includedin a set of the recognition suitable data. According to the presentinvention, the number of the analysis indices is a total of ten asfollows: a luminosity mean, a luminosity variance, a black-and-whitepixel percentage, a segmentation luminosity mean, a segmentationluminosity variance, a segmented black-and-white pixel percentage, thenumber of segmented pixels, a minutia segment length, a minutia segmentratio and a minutia angle. The image analysis learning unit 317 searchesa set of the recognition suitable data for arbitrary image data, andcalculates all ten analysis indices by using searched image data.Additionally, the image analysis learning unit 317 allows the calculatedanalysis indices to be stored in such a manner that the calculatedanalysis indices correspond to a set of the recognition suitable data.

In step 703, the image analysis learning unit 317 searches for a set ofthe recognition unsuitable data which has been previously stored. Afterthat, the image analysis learning unit 317 calculates the analysisindices by using all of the image data included in a set of therecognition unsuitable data. According to the present invention, thenumber of the analysis indices is a total of ten as follows: aluminosity mean, a luminosity variance, a black-and-white pixelpercentage, a segmentation luminosity mean, a segmentation luminosityvariance, a segmented black-and-white pixel percentage, the number ofsegmented pixels, a minutia segment length, a minutia segment ratio, anda minutia angle. The image analysis learning unit 317 searches a set ofthe recognition unsuitable data for arbitrary image data, and cancalculate all ten analysis indices by using searched image data.Additionally, the image analysis learning unit 317 allows the calculatedanalysis indices to be stored in such a manner that the calculatedanalysis indices correspond to a set of the recognition unsuitable data.

In step 705, the image analysis learning unit 317 searches for all ofthe analysis indices preserved in manner that the analysis indicescorrespond to a set of the recognition suitable data, and calculates amean, a covariance and a distribution of each of the searched analysisindices. Next, the image analysis learning unit 317 preserves acalculated mean, covariance and distribution thereof. For instance, ifthe luminosity variances are included in the analysis indices calculatedin step 701, the image analysis learning unit 317 searches for theluminosity variance corresponding to each of image data which is a setof recognition suitable data, and calculates a mean of searchedluminosity variances. Further, the image analysis learning unit 317calculates a covariance of the luminosity variances by using thesearched luminosity variances and by using the calculated mean of theluminosity variances. The image analysis learning unit 317 thencalculates a distribution of the searched luminosity variances, andstores the calculated mean, the covariance and the distribution of theluminosity variances. The mean of each of the analysis indices of a setof the recognition suitable data is represented as μ_(s,i). The varianceof each of the analysis indices of a set of the recognition suitabledata is represented as σ_(s,i) ². The mean of each of the analysisindices of a set of the recognition unsuitable data is represented asμ_(f,i). The variance of each of the analysis indices of a set of therecognition unsuitable data is represented as σ_(f,i) ².

The image analysis learning unit 317 searches for each of the analysisindices preserved in the manner that the analysis indices correspond toa set of the recognition unsuitable data, and calculates the mean, thecovariance, and the distribution.

In step 707, the image analysis learning unit 317 calculates thereference values by using the distribution of the analysis indicesstored in step 705. For example, when it is assumed that the analysisindices of the recognition suitable data follow a normal distribution,the image analysis learning unit 317 can draw the reference values byusing a method of maximum likelihood, and can set up the drawn referencevalues.

The method of maximum likelihood is formulated as follows.

$p\left( {x_{i}\left. H_{s} \right)\begin{matrix}\begin{matrix}\begin{matrix}D_{1} \\ > \end{matrix} \\ < \end{matrix} \\D_{0}\end{matrix}{p\left( x_{i} \right.}H_{F}} \right)$

In the above expression, D₁ represents a determination that the imagedata is suitable for image recognition, and D₀ represents adetermination that the image data is unsuitable for image recognition.Herein, p(x_(i)|H_(S)) represents a distribution of the i-th analysisindices of a set of the recognition suitable data, and p(x_(i)|H_(F))represents a distribution of the i-th analysis indices of a set of therecognition unsuitable data.

The image analysis learning unit 317 can draw as a reference value apart in which a distribution of the i-th analysis indices of a set ofthe recognition suitable data is the same as that of the i-th analysisindices of a set of the recognition unsuitable data, and set the samepart to the reference value. For instance, when drawing the referenceluminosity variance corresponding to the reference value of theluminosity variance, the image analysis learning unit 317 searches for adistribution of the luminosity variances of a set of the storedrecognition suitable data and a distribution of the luminosity variancesof a set of the stored recognition unsuitable data. Then, the imageanalysis learning unit 317 searches for the same parts between thedistribution of the luminosity variances of a set of the recognitionsuitable data and the distribution of the luminosity variances of a setof the recognition unsuitable data, and sets up the reference luminosityvariances with the searched parts. More specifically, each of thedistribution of the luminosity variances of a set of the recognitionsuitable data and of the distribution of the luminosity variances of aset of the recognition unsuitable data is a normal distribution. Thus,each of a distribution curve of the luminosity variances of a set of therecognition suitable data and of a distribution curve of the luminosityvariances of a set of the recognition unsuitable data is bilaterallysymmetric with respect to the mean placed in the middle of a bell curve.If it is expressed in one coordinate system the distribution curves ofthe luminosity variances of a set of the recognition suitable data andof a set of the recognition unsuitable data, the curves will cross eachother at any one point. The X-axis of the coordinate system can be theluminosity variance, and the Y-axis thereof can be the distribution. Atthis time, a value of the luminosity variance at which the two curvescross each other can be a value of the reference luminosity variance.

The image analysis learning unit 317 repeats the above procedures todraw the reference value of each of the analysis indices, and sets drawnreference values.

The control unit 201 controls the image recognition reliability learningunit 309 of the image processing unit 205 to calculate the referencereliability required to perceive whether image recognition is suitablyperformed, and sets a calculated reference reliability. The process inwhich the image recognition reliability learning unit 309 draws thereference reliability is the same as a process in which the imageanalysis learning unit 317 draws the reference values. Morespecifically, the image recognition reliability learning unit 309calculates the recognition reliability by using each of the image dataof a set of the reliability suitable data. The image recognitionreliability learning unit 309 then calculates a mean and a distributionof the recognition reliabilities by using the calculated recognitionreliabilities. Also, the image recognition reliability learning unit 309calculates a mean and a distribution of the recognition reliabilitiesfrom each of the image data of a set of the reliability unsuitable data.It will now be assumed that each of the distribution of a set of thereliability suitable data and the distribution of a set of thereliability unsuitable data is normal distribution. By using the methodof maximum likelihood, the image recognition reliability learning unit309 sets to the reference reliability a recognition reliability of apart in which the distribution of a set of the reliability suitable datais the same as the distribution of a set of the reliability unsuitabledata.

The method of maximum likelihood is formulated as follows.

$p\left( {c\left. H_{s} \right)\begin{matrix}\begin{matrix}\begin{matrix}D_{1} \\ > \end{matrix} \\ < \end{matrix} \\D_{0}\end{matrix}{p\left( c \right.}H_{F}} \right)$

In the above expression, D₁ represents a determination that the imagedata is suitable for image recognition, and D₀ represents adetermination that the image data is unsuitable for image recognition.Herein, p(c|H_(s)) represents a distribution of the recognitionreliability of a set of the reliability suitable data, and p(c|H_(F))represents a distribution of the recognition reliability of a set of thereliability unsuitable data. The image recognition reliability learningunit 309 can draw the reference reliability from a part in which adistribution of the recognition reliability of a set of the reliabilitysuitable data is the same as that of the recognition reliability of aset of the reliability unsuitable data, and set the referencereliability to the same part.

With reference to FIG. 2, the control unit 201 controls the unitsconstituting the imaging system according to the present invention inorder to perform the function of image recognition. Particularly, thecontrol unit 201 controls the image processing unit 205 and the imageanalyzing unit 207 to perform the function of image recognition. Whenthe user requests image recognition, the control unit 201 receives theimage data from the camera unit 209 or from the communication unit 203to provide an output thereof to the illumination analyzing unit 311 ofthe image analyzing unit 307. At this time, the control unit 201controls the illumination analyzing unit 311 to analyze the illuminationstate. Then, the control unit 201 provides the image data to thepreprocessing unit 301 of the image processing unit 205, and controlsthe preprocessing unit 301 to preprocess the image data. Subsequently,the control unit 201 controls the image segmenting unit 303 to segmentthe image data including only a region-of-interest.

The control unit 201 provides segmented image data to the illuminationanalyzing unit 311, and controls the illumination analyzing unit 311,the angle analyzing unit 313 and the size analyzing unit 315 tocalculate analysis indices which indicate a illumination state, an anglestate of the object of recognition, and a size state. Next, the controlunit 201 controls the image recognition analyzing unit 319, and compareseach of the analysis indices with a previously stored reference value.If the analysis indices do not entirely meet the requisites for imagerecognition, the control unit 201 perceives a result, corresponding toeach of the analysis indices which don't meet the requisites, to be thecause of the recognition error, and informs the user of the cause. Ifthe analysis indices entirely meet the requisites for image recognition,the control unit 201 provides the segmented image data to the featureextracting unit 305. Subsequently, the control unit 201 controls thefeature extracting unit 305 to extract feature image data from thesegmented image data. Further, the control unit 201 controls the imagerecognizing unit 307, performs the function of image recognition byusing the feature image data of the segmented image data, and calculatesa recognition reliability of a performance result to determine whetherthe feature image data of the segmented image data is suitable for imagerecognition through the calculated recognition reliability. If it isdetermined that the feature image data of the segmented image data isunsuitable for image recognition, the control unit 201 controls theimage recognizing 307 to perceive the cause of the recognition error,and informs the user of the perceived cause.

Before the user requests image recognition, the control unit 201controls the image analysis learning unit 317 to set reference valuesneeded to perceive whether the reference values are suitable for imagerecognition. Then, the control unit 201 controls the image recognitionreliability learning unit 309 to set the reference reliability needed toperceive whether the reference reliability is suitable for imagerecognition.

FIG. 4 illustrates an operation for performing a function of imagerecognition with image data according to the present invention.Referring to FIGS. 2 to 4, a procedure of executing image recognition byusing the image data will be described.

In step 401, if it is determined that image recognition is requested bythe user, the procedure proceeds to step 403. Otherwise, if it isdetermined that image recognition is not requested by the user, theprocedure repeats step 401.

In step 403, the illumination state of the image data is analyzed by theimaging system. In step 405, the imaging system eliminates noise fromthe image data or adjusts the luminosity of the image data, so that theimage data can be improved in order to be suitable for imagerecognition. Thereafter, the imaging system establishes with the imagedata a region-of-interest including the object of recognition, andsegments the image data including only a set region-of-interest. In step407, the imaging system analyzes the illumination state and the sizestate of the segmented image data, and the angle state of the object ofrecognition. Then, the imaging system determines whether the segmentedimage data is suitable for image recognition through the analyzedresults. If it is determined that the segmented image data is suitablefor image recognition through the analyzed results, the procedureproceeds to step 409. If it is determined that the segmented image datais unsuitable for image recognition, the imaging system perceives thecause of the recognition error, and informs the user of the cause.

In step 409, the imaging system performs the function of imagerecognition by using the segmented image data, and calculates therecognition reliability of a performed outcome. Next, the imaging systemdetermines whether the segmented image data is suitable for imagerecognition through the recognition reliability. If it is determinedthat the segmented image data is suitable for image recognition throughthe recognition reliability, the imaging system carries out the nextremaining functions. If it is determined that the segmented image datais unsuitable for image recognition, the imaging system perceives thecause of the recognition error, and informs the user of the cause.

FIG. 5 illustrates an operation for informing a user of a cause of theimage recognition error by perceiving the cause of the image recognitionerror with the image data of according to the present invention. Withreference to FIGS. 2 to 5, a description will be given to a procedure ofinforming the user of the cause of the image recognition error when theimaging system perceives the cause of the image recognition error.

If image recognition is requested by the user in step 501, the controlunit 201 proceeds to step 503. If image recognition is not requested bythe user, the procedure repeats step 501.

In step 503, the control unit 201 receives image data required toperform the process of image recognition and outputs the image data tothe illumination analyzing unit 311 of the image analyzing unit 207. Atthis time, the control unit 201 controls the illumination analyzing unit311 to analyze the illumination state of the image data by calculatinganalysis indices. Thereafter, the control unit 201 stores calculatedanalysis indices. According to this embodiment the present invention,the analysis indices corresponding to the illumination state may bedesignated only as the luminosity mean, the luminosity variance, and thepercentage of the number of black-and-white pixels. The entire imagedata may be brighter than a first value or darker than a second valuedepending on the illumination state. Furthermore, if a part of the imagedata is bright, whereas another part of the image data is dark, thenluminosity contrast can occur more than a prescribed value. As imagedata is not suitable for image recognition, the illumination analyzingunit 311 analyzes the luminosity and the luminosity contrast of thereceived image data, and also analyzes whether backlight of theillumination occurs.

Therefore, the illumination analyzing unit 311 calculates the luminositymean, the luminosity variance, and the percentage of the number ofblack-and-white pixels of the image data to analyze the illuminationstate. For example, when the image data is constructed with 4×4 pixels,and when the range of the luminosity is 3 bits, the illuminationanalyzing unit 311 can calculate the luminosity mean if the luminositiesof the pixels are sequentially 3, 4, 5, 2, 1, 3, 4, 4, 2, 3, 3, 2, 5, 3,3, and 3, respectively. The above-mentioned luminosity mean equals(50/16)=3.125. As it is usual that the image of the object ofrecognition places in the center of the picture image, the illuminationanalyzing unit 311 can calculate the luminosity mean by giving a weightto the luminosity value of the 2×2 pixels. Among the above-mentionedluminosity values corresponding to the 4×4 pixels, to evaluate the meanby multiplying weight 2 by the luminosity value corresponding to thecenter pixels, the mean equals 3.9375 from the calculation expression asfollows.(3+4+5+2+1+(3×2)+(4×2)+4+2+(3×2)+(3×2)+2+5+3+3+3)/16=3.9375

Next, the illumination analyzing unit 311 calculates the luminosityvariance by using the luminosity values and luminosity mean of the imagedata. The luminosity variance represents the luminosity contrast of theimage data. For instance, if the luminosity mean equals 4 at presentwhile using the above luminosity values, the luminosity variance equals1.875 from the calculation expression as follows.(1+0+1+4+9+1+0+0+4+1+1+4+1+1+1+1)/16=1.875

The illumination analyzing unit 311 calculates the percentage of thepixels respectively corresponding to white and black among theluminosities respectively corresponding to pixels of the image data.Thus, the white pixels or the black pixels are produced more than usualimage data among the image data generated when the illumination is in astate of backlight. Therefore, the illumination analyzing unit 311calculates the number of black-and-white pixels of the image data andthen the percentage of the number of black-and-white pixels per thetotal number of pixels. For example, when the image data constructedwith 4×4 pixels has a luminosity range of 3 bits, a luminosity of 0corresponding to white, and a luminosity of 7 corresponding to black,the illumination analyzing unit 311 can calculate the number ofblack-and-white pixels of 8 by adding the number of pixels of 4 throughthe luminosity of 7 to the number of pixels of 4 through the luminosityof 0. Next, the illumination analyzing unit 311 calculates thepercentage of the number of black-and-white pixels obtained from amongthe total number of pixels as follows.8/16×100=50[%]Subsequently, the control unit 201 stores the currently calculatedluminosity mean, luminosity variance, and percentage of the number ofblack-and-white pixels of the image data.

In step 505, the control unit 201 provides the image data to thepreprocessing unit 301 of the image processing unit 205. The controlunit 201 controls the preprocessing unit 301 and image segmenting unit303 to improve or segment the image data to produce image data suitablefor image recognition. Under the control of the control unit 201, thepreprocessing unit 301 eliminates the noise of the image data orimproves the data to have a brightness suitable for image recognition,and outputs them to the image segmenting unit 303. Under the control ofthe control unit 201, the image segmenting unit 303 sets theregion-of-interest including the object of recognition among the imagedata, and produces the segmented image data including only theregion-of-interest from the image data.

In step 507, the control unit 201 provides the segmented image data tothe illumination analyzing unit 311 of the image analyzing unit 207.Next, the control unit 201 controls the illumination analyzing unit 311to calculate the analysis indices to analyze the illumination state ofthe segmented image data. Then, the control unit 201 stores thecalculated analysis indices. According to this embodiment, the analysisindices corresponding to the illumination state are designated only thesegmentation luminosity mean, the segmentation luminosity variance andthe number of segmented black-and-white pixels.

When analyzing the illumination state of the segmented image data, theillumination analyzing unit 311 calculates the segmentation luminositymean, the segmentation luminosity variance, and the percentage of thenumber of segmented black-and-white pixels in the same scheme by whichit has analyzed the illumination state of the image data in step 503.For instance, if the segmented image data constructed with 2×2 pixelshas the range of the luminosity of 3 bits, and has the luminosities ofthe pixels of 3, 4, 3, and 3, the illumination analyzing unit 311 cancalculate the segmentation luminosity mean as 3.25 from such acalculation expression as (13/4)=3.25. Then, the illumination analyzingunit 311 calculates the segmentation luminosity variance by usingluminosities and the calculated segmentation luminosity mean of thesegmented image data. For example, if the segmentation luminosity meanequals 3, the segmentation luminosity variance can be evaluated fromsuch a calculation expression as (0+1+0+0)/4=0.25. Subsequently, theillumination analyzing unit 311 computes the percentage of the number ofthe pixels respectively corresponding to black and white among theluminosities respectively corresponding to the pixels of the segmentedimage data. For example, when the image data constructed with 2×2 pixelshas the range of the luminosity of 3 bits, has the luminosity of 0corresponding to white, and has the luminosity of 7 corresponding toblack, if the luminosities of the pixels are sequentially 3, 7, 4, and0, respectively, the illumination analyzing unit 311 can compute thenumber of black-and-white pixels of 2 by adding the number of pixels of1 through the luminosity of 7 to the number of pixels 1 through theluminosity of 0. Then, the illumination analyzing unit 311 can evaluatethe percentage of the number of segmented black-and-white pixels amongthe total number of pixels from such a calculation expression as2/4×100=50[%]. Next, the control unit 201 stores the currentlycalculated segmentation luminosity mean, segmentation luminosityvariance, and percentage of the number of segmented black-and-whitepixels of the segmented image data.

In step 509, the control unit 201 outputs the segmented image data tothe size analyzing unit 315. Subsequently, the control unit 315 controlsthe size analyzing unit 315 to calculate the analysis indices to analyzethe size state of the segmented image data. Next, the control unit 201stores the calculated analysis indices. According to this embodiment,the analysis index corresponding to the size state of the segmentedimage data is designated the number of image pixels.

If the size of the segmented image data is equal to or less than areference value, the control unit 201 cannot extract feature image datafrom the segmented image data. Accordingly, the control unit 201controls the size analyzing unit 315 to compute the size of thesegmented image data, and allows a computed size to be stored. Forinstance, the size analyzing unit 315 can evaluate the size of thesegmented image data as the number of pixels. Herein, the number ofpixels is referred to as “the number of image pixels.”

In step 511, the control unit 201 provides the segmented image data tothe angle analyzing unit 313. Thereafter, the control unit 313 controlsthe angle analyzing unit 313 to compute the analysis indices to analyzethe angle state of the object of recognition included in the segmentedimage data. As the image data is usually data related to two-dimensionalimages, the angle state of the object of recognition causes cases inwhich the angle analyzing unit 313 cannot extract the feature image datarequired to perform the process of image recognition. Thus, beforeextracting the feature image data from the segmented image data, thecontrol unit 201 enables the angle analyzing unit 313 to calculate theanalysis indices corresponding to the angle state of the object ofrecognition, and stores the calculated analysis indices. According tothis embodiment of the present invention, the analysis indices accordingto the angle state of the object of recognition are designated theminutia segment length, the minutia segment ratio and the minutia angle.

When analyzing the angle state of the object of recognition, the angleanalyzing unit 313 extracts more than one minutia of the object ofrecognition, and computes the angle of the object of recognition byusing the extracted minutiae. When a procedure for calculating a faceangle is executed in the process of a face recognition which is a typeof image recognition, when the eyes and mouth have been alreadydesignated minutiae, on receiving the segmented image data, the angleanalyzing unit 313 is able to detect the eyes and mouth from face imagedata. Then, the angle analyzing unit 313 can express the coordinates ofthe detected eyes as an X coordinate and an Y coordinate, and canlikewise express a coordinate of the detected mouth. Herein, the angleanalyzing unit 313 bisects the detected two X coordinates of the twoeyes, and can calculate a central X coordinate. Next, the angleanalyzing unit 313 computes the lengths of a segment between the twoeyes and of a segment between the left eye and the mouth. Herein, thesegment between the two eyes, the segment between the left eye and themouth, and a segment between the right eye and the mouth are referred toas a first, a second and a third segment, respectively.

Because the face inclined on the left or on the right is unsuitable forimage recognition, the angle analyzing unit 313 calculates thedifference between the calculated central X coordinate between the twoeyes and the calculated X coordinate of the mouth, which is referred toas “a minutia segment length.” If the face is inclined upwards ordownwards, even though the face is not inclined on the left or on theright, it is unsuitable for image recognition. For these reasons, theangle analyzing unit 313 calculates the length ratio of the firstsegment between the two eyes to the second segment between the left eyeand the mouth, which is referred to as a minutia segment ratio.Subsequently, the angle analyzing unit 313 calculates a minutia angleformed between the first segment and a set horizontal line. The angleanalyzing unit 313 designates the minutiae of the object of recognition,and calculates the minutia segment length, the minutia segment ratio,and the minutia angle in terms of the designated minutiae. The controlunit 201 stores the calculated minutia segment length, minutia segmentratio, and minutia angle.

In step 513, the control unit 201 controls the image recognizing unit319 to determine whether the analysis indices are suitable for imagerecognition, wherein the analysis indices include the illumination stateof the image data, the illumination state and the size state of thesegmented image data, and the angle state of the object of recognition.If the control unit 201 receives from the image recognition analyzingunit 319 a resultant determination that the segmented image data issuitable for image recognition, then the procedure proceeds to step 515.If it is determined that the segmented image data is unsuitable forimage recognition, the control unit 201 perceives the cause of the imagerecognition error, and the procedure proceeds to step 521.

When perceiving whether the image data is suitable for imagerecognition, the image recognition analyzing unit 319 uses the indexindices stored in steps 503, 507, 509 and 511.

Particularly, the image recognition analyzing unit 319 searches for theluminosity mean, the luminosity variance and the percentage of thenumber of black-and-white pixels of the image data stored in step 503.Then, the image recognition analyzing unit 319 checks whether theluminosity mean lies between a minimum luminosity mean and a maximumluminosity mean, both of which are designated by the image recognitionlearning unit 317. The image recognition analyzing unit 319 checkswhether the luminosity variance is equal to or less than the referenceluminosity variance designated by the image recognition learning unit317. Subsequently, the image recognition analyzing unit 319 perceiveswhether the percentage of the number of black-and-white pixels is equalto or less than the percentage of reference pixels designated by theimage recognition learning unit 317.

After that, the image recognition analyzing unit 319 searches for thesegmentation luminosity mean, the segmentation luminosity variance, andthe percentage of the number of black-and-white pixels of the segmentedimage data preserved in step 507. Then, the image recognition analyzingunit 319 checks whether the segmentation luminosity mean lies betweenthe minimum segmentation luminosity mean and the maximum segmentationluminosity mean, both of which are designated the image recognitionlearning unit 317. Subsequently, the image recognition analyzing 319checks whether the segmentation luminosity variance is equal to or lessthan the reference segmentation luminosity variance designated by theimage analysis learning unit 317. Next, the image recognition analyzingunit 319 perceives whether the percentage of the number of the segmentedblack-and-white pixels is equal to or less than the percentage of thesegmented reference pixels designated by the image recognition learningunit 317.

Subsequently, the image recognition analyzing unit 319 searches for thenumber of image pixels of segmented image data preserved in step 509.Then, the image recognition analyzing unit 319 checks whether the numberof the image pixels is equal to or less than the number of the referencepixels designated by the image analysis learning unit 317. Next, theimage recognition analyzing unit 319 searches for the minutia segmentlength, minutia segment ratio, and minutia angle of the segmented imagedata stored in step 511. Thereafter, the image recognition analyzingunit 319 perceives whether the minutia segment length is equal to orshorter than the reference minutia segment length designated by theimage analysis learning unit 317. Then, the image recognition analyzingunit 319 perceives whether the minutia segment ratio lies between theminimum minutia segment ratio and the maximum minutia segment ratio,both of which are designated by the image analysis learning unit 317. Inaddition, the image recognition analyzing unit 319 perceives whether theminutia angle is equal to or smaller than the reference minutia angledesignated by the image analysis learning unit 317.

The image recognition analyzing unit 319 produces data for perceivingthe recognition error (hereinafter, recognition error perception data)including if the above-mentioned ten recognition conditions aresatisfied to provide the recognition error perception data to thecontrol unit 201. Then, the control unit 201 receives the recognitionerror perception data in order to analyze them. Next, depending on aresultant analysis, if it is determined that the ten recognitionconditions are completely satisfied, the control unit 201 proceeds tostep 515. If any of the ten recognition conditions are not satisfied,then the control unit 201 perceives that the unsatisfied recognitionconditions are the causes of the recognition errors, and the controlunit 201 proceeds to step 521. For example, if the segmentationluminosity variance of the segmented image data out of the tenrecognition conditions is equal to or greater than the referencesegmentation luminosity variance, the image recognition analyzing unit319 produces recognition error perception data containing information onthe above dissatisfaction of the segmentation luminosity variance, andoutputs them to the control unit 201. Next, the control unit 201analyzes the recognition error perception data to perceive that aluminosity contrast indicated by the segmentation luminosity variance isa cause of the recognition error, and proceeds to step 521.

In step 515, the control unit 201 controls the feature extracting unit305 of the image processing unit 205 to extract singular feature imagedata which is able to cause the object of recognition to bedistinguished from the segmented image data. Subsequently, the controlunit 201 controls the image recognizing unit 307 to perform the functionof image recognition. Specifically, the image recognizing unit 307searches for the recognition data which has been previously stored.Then, the image recognizing unit 307 extracts the feature image datafrom the image data of the recognition data. Subsequently, the imagerecognizing unit 307 compares the feature image data of the image dataof the recognition data with the feature image data extracted from thesegmented image data, and computes the recognition reliability of imagerecognition. Further, the image recognizing unit 307 sets as arepresentative recognition reliability the recognition reliabilityhaving a higher value than any other calculated recognition reliability.For instance, the image recognizing unit 307 compares the tone of colorof the feature image data of the segmented image data with the tone ofcolor of the feature image data of the image data stored in advance,checks whether the two tones of color thereof agree with each other, andis then able to calculate the recognition reliability. Specifically, ifthe feature image data of the segmented image data, constructed with 2×2pixels, has a color tone bit of one bit, color tone bits of pixels ofthe feature image data such as 1, 0, 1, and 0, and color tone bits ofpixels of the feature image data of the image data stored in advancesuch as 1, 0, 0, and 0, then the control unit 201 can compute arecognition reliability of 75[%].

In step 517, the control unit 201 controls the image recognizing unit307 to perceive whether the performance of image recognition hasresulted in a reliable outcome. The image recognizing unit 307 comparesthe representative recognition reliability in step 515 with a referencereliability designated by the image recognition reliability learningunit 309, produces reliability data including an outcome resulting fromthe comparison, and outputs the reliability data to the control unit201. Subsequently, the control unit 201 receives and analyzes thereliability data from the image recognizing unit 307. Depending on theresult of the analysis, if the representative recognition reliability isequal to or greater than the reference reliability, the control unit 201proceeds to step 519. If not, then the control unit 201 perceives thecause of, the recognition error, and proceeds to step 521.

In order to perceive the cause of the recognition error, the controlunit 201 uses an algorithm of a Mahalanobis distance. The control unit201 searches for the ten analysis indices calculated in steps 503, 507,509, and 511, evaluates analysis indices of the analysis suitablelearning data corresponding to the ten analysis indices, searches forthe stored means and standard deviations, and calculates the Mahalanobisdistance. The control unit 201 searches for the longest of thecalculated Mahalanobis distances, perceives to be the cause of therecognition error each of the environmental elements corresponding tothe analysis indices substituted for evaluating a searched Mahalanobisdistance, and proceeds to step 521.

An analysis index maximizing the Mahalanobis distance can be obtainedfrom an expression defined as follows:

$\begin{matrix}{i = {\arg\mspace{14mu}\max{\frac{x_{j} - \mu_{s,j}}{\sigma_{s,j}}}}} \\{{j = 1},2,\ldots\mspace{14mu},N}\end{matrix}$

In the above expression, x_(j) is a variable representing an analysisindex required to perceive the suitability of image recognition. Forx_(j), any of the luminosity mean, the luminosity variance, thepercentage of black-and-white pixels, the segmentation luminosity mean,the segmentation luminosity variance, the percentage of segmentedblack-and-white pixels, the number of segmented pixels, the minutiasegment length, the minutia segment ratio, and the minutia angle may besequentially substituted. μ_(s,j) and σ_(s,j) represent valuesdesignated by the image analysis learning unit 317. μ_(s,j) is avariable representing a mean corresponding to the analysis index.σ_(s,j) is a variable representing a standard deviation corresponding tothe analysis index. For instance, when the luminosity mean issubstituted for x_(j), for μ_(s,j), a mean calculated from luminositymeans of the image data of the reliability suitable data is substituted,and for σ_(s,j), a standard deviation of the luminosity means of theimage data of the reliability suitable data is substituted. Next, thecontrol unit 201 computes the Mahalanobis distance by using thesubstituted values. The control unit 201 computes the Mahalanobisdistance by using all of the substituted analysis indices, and searchesthe computed Mahalanobis distances for the longest Mahalanobis distance.The control unit 201 searches for the analysis indices used to computethe searched Mahalanobis distance, and perceives an environmental factorcorresponding to each of the analysis indices to be the cause of theimage recognition error.

In step 521, the control unit 201 informs the user of the causes of theimage recognition errors. For instance, if the control unit 201perceives that the cause of the image recognition error is incurred bythe brightness of the illumination, the display unit 213 displays amessage such as, “The current illumination is too bright! Please go to adarker place and try performing image recognition again!,” or informsthe user of the message in the manner of an acoustic output.

FIG. 6 illustrates the conception of a picture image according to thepresent invention. With reference to FIGS. 1 to 6, it will be supposedthat the imaging system according to an embodiment of the presentinvention images a human face as the object of recognition.

When the user requests image recognition, the imaging system receivesimage data 601. The imaging system computes an analysis indexcorresponding to an illumination state by using the image data 601. Theimaging system produces from the image data 601 segmented image data 603including only the face corresponding to a region-of-interest, andcalculates from the segmented image data 603 the coordinates of the eyesand mouth corresponding to minutiae of the face recognition. From thesegmented image data 603, the imaging system of the present inventioncomputes an analysis index corresponding to an illumination state, ananalysis index corresponding to a size state, and an analysis indexcorresponding to angle state of the face corresponding to the object ofrecognition. The imaging system compares the analysis indices computeduntil now with reference values of the analysis indices set by anoperation of the image analysis learning unit 317, and checks whether anoutcome resulting from the comparison satisfies each of recognitionsuitable requisites.

If the analysis indices are completely satisfied with the recognitionsuitable requisites, then the imaging system of the present inventionextracts feature image data from the segmented image data 603. Theimaging system compares the extracted feature image data with featureimage data of the image data registered in advance in order to calculatea recognition reliability. Subsequently, the imaging system compares thecalculated recognition reliability with a reference reliability set byan operation of the image recognition reliability learning unit 309. Ifthe calculated recognition reliability is equal to or greater than thereference reliability, the imaging system perceives that imagerecognition is suitably performed, and stores a result of imagerecognition. If not, then the imaging system perceives that imagerecognition is unsuitably performed. Next, the imaging system informsthe user of the causes of the recognition errors.

If each of the analysis indices is not satisfied with any of therecognition suitable requisites, the imaging system perceives that theunsatisfied recognition suitable requisites are the causes of therecognition errors, and informs the user that the unsatisfiedrecognition suitable requisites cause the recognition errors.

If the size state of the image data 601 and the angle state of theobject of recognition are suitable for image recognition, but theillumination state of the image data 601 is unsuitable for imagerecognition, when the imaging system checks whether image recognition isperformed suitably through the analysis index of the illumination state,it perceives that the analysis index of the illumination state is notsatisfied with the recognition suitable requisite. Hence, the imagingsystem perceives that the illumination state of the image data 601 isthe cause of the image recognition error, and informs the user with amessage such as, “Please adjust the current state of illumination, andtry performing image recognition again!”

If the states of illumination and size of the image data 601 and theangle state of the object of recognition are completely suitable forimage recognition, the imaging system determines that the analysisindices of the illumination state, of the size state, and of the anglestate of the object of recognition satisfy the recognition suitablerequisites. In succession, the imaging system compares the feature imagedata of the segmented image data 603 with the feature image data of theimage data stored in advance, and calculates the recognitionreliability. If the calculated recognition reliability is equal to orgreater than the reference reliability, the imaging system stores theresult of image recognition. If not, then the imaging system perceivesthe causes of the image recognition errors, and can inform the user ofthe causes thereof.

If the states of illumination and size of the image data 605 aresuitable for image recognition, the angle state of the object ofrecognition is unsuitable for image recognition, the imaging systemperceives that the analysis index of the angle state of the object ofrecognition is not satisfied with the recognition suitable requisite. Asa result, the imaging system perceives that the angle state of theobject of recognition of the segmented image data 607 is the cause ofthe image recognition error, and informs the user with a message, suchas “Please adjust the face angle corresponding to the angle state of theobject of recognition, and try performing image recognition again!”

In the process of performing image recognition as described above, theimaging system can perceive the causes of the image recognition errors,inform the user of the perceived causes thereof, and enable the user toperform more efficient image recognition functions.

While the invention has been shown and described with reference to acertain preferred embodiment thereof, it will be understood by thoseskilled in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the invention.For instance, the present invention not only applies image recognitionto the face recognition but also emphasizes the recognition of the faceof a human being. However, image recognition thereof can be applied toobjects other than the face of the human being. Therefore, the spiritand scope of the present invention must be defined not by describedembodiments thereof but by the appended claims and equivalents of theappended claims.

1. A method for informing a user of an image recognition error in animaging system, the method comprising the steps of: (1) generating imagedata by taking a picture of an object of recognition if imagerecognition is requested by a user; (2) calculating analysis indicescorresponding to environmental factors by detecting from the image datathe environmental factors that may cause the image recognition errors;(3) determining whether the analysis indices are suitable for imagerecognition by checking whether each of the analysis indices is includedin or excluded from a normal range of reference values; and (4)informing the user of the cause of the image recognition error afterperceiving that each of the environmental factors corresponding to theanalysis indices excluded from the normal range of the reference valuesis a cause of the image recognition error if any of the analysis indicesare excluded from the normal range of the reference values, wherein theenvironmental factors include at least one among an illumination stateand a size state of the image data, and an angle state of the object ofrecognition.
 2. The method as claimed in claim 1, further comprising:searching previously stored image data for the image data included inthe object of recognition of the image data if all of the analysisindices are included in the normal range of the predetermined referencevalues; calculating a recognition reliability corresponding to thereliability of an outcome resulting from comparing searched image datawith the image data; determining whether image recognition issuccessfully performed by comparing the recognition reliability with areference reliability; informing the user of the success of imagerecognition if the recognition reliability is equal to or greater thanthe reference reliability; and informing the user of the cause of therecognition error by rechecking the cause of the recognition errorthrough the analysis indices if the recognition reliability is less thanthe reference reliability.
 3. The method as claimed in claim, whereinthe reference values are calculated in terms of a method of maximumlikelihood.
 4. The method as claimed in claim 1, wherein step (4)further comprises: searching for at least one analysis index excludedfrom the normal range of the reference values; designating theenvironmental factors corresponding to the analysis indices as causes ofthe recognition errors; and informing the user of each of the designatedenvironmental factors as the cause of the errors.
 5. The method asclaimed in claim 1, wherein the reference reliability is calculated interms of a method of maximum likelihood.
 6. The method as claimed inclaim 1, wherein step (4) further comprises: calculating a value of aMahalanobis distance by substituting each of the analysis indices for anexpression of the Mahalanobis distance; searching the calculated valuesof the Mahalanobis distances for a longest value thereof; designatingthe environment factor corresponding to the searched analysis index asthe cause of the recognition error after searching for the analysisindex corresponding to the searched longest value; and informing theuser of the designated environment factor as the cause of therecognition error.
 7. An apparatus for informing a user of an imagerecognition error in an imaging system, the apparatus comprising: animage analyzing unit for calculating analysis indices corresponding toenvironmental factors by detecting from input image data theenvironmental factors that cause the image recognition errors, foroutputting a cause of the image recognition error after perceiving thateach of the environmental factors corresponding to the analysis indicesexcluded from the normal range of reference values is the cause of theimage recognition error if any of the analysis indices are excluded froma normal range of the reference values, and for outputting the imagedata to an image processing unit if all of the analysis indices areincluded in the normal range of the reference values; an imageprocessing unit for searching previously stored image data for imagedata including an object of recognition of the image data, forcalculating a recognition reliability corresponding to the reliabilityof an outcome resulting from comparing the searched image data with theimage data, and for outputting the cause of the recognition error afterrechecking the cause of the error through the analysis indices if therecognition reliability is less than a reference reliability; and acontrol unit for controlling the image analyzing unit and the imageprocessing unit if image recognition is requested by the user, and forinforming the user of the cause of the recognition error provided by theimage analyzing unit or by the image processing unit wherein the imageanalyzing unit calculates the analysis indices corresponding to theenvironmental factors including at least one among an illumination stateand a size state of the image data, and an angle state of the object ofrecognition.
 8. The apparatus as claimed in claim 7, wherein the imageanalyzing unit calculates the reference values in terms of a method ofmaximum likelihood.
 9. The apparatus as claimed in claim 7, wherein theimage processing unit calculates the reference reliability in terms of amethod of maximum likelihood.
 10. The apparatus as claimed in claim 7,wherein the image processing unit calculates a value of a Mahalanobisdistance by substituting each of the analysis indices for an expressionof the Mahalanobis distance, searches the calculated values of theMahalanobis distances for a longest value thereof, and perceives thatthe environment factors corresponding to the searched analysis indicesare the causes of the recognition error.